Multitouch Gesture Learning and Recognition System

We build a system that learns and recognizes multitouch gestures – movements of human fingers on a multitouch surface. We form an example vocabulary of gestures that demonstrates the subtleties of finger movements that can be recognized. A video processing system extracts a feature set representing these movements. We recognize the spatial arrangement of fingers in a gesture to address issues regarding the preprocessing of the feature set for consistency across samples of a gesture. Hidden Markov Models (HMMs) are then used to learn gestures by example, and then consistently recognize them. We present an evaluation that demonstrates the robustness of the methodology.